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The idea of diffractive processes with a hard scale involved, to resolve the underlying parton 
dynamics, was presented at the first Blois conference in 1985 and experimentally verified a few 
years later. Today hard diffraction is an active research field with high-quality data and new 
theoretical models. The trend from Regge-based pomeron models to QCD-based parton level 
models has given insights on QCD dynamics involving perturbative gluon exchange mecha- 
nisms, including the predicted BFKL-dynamics, as well as novel ideas on non-perturbative 
colour fields and their interactions. 



1 Idea and discovery of hard diffraction 

At the first Blois meeting in 1985 we presented^ our novel idea'^ to consider a hard scale in 
diffractive scattering to resolve an underlying parton level interaction and thereby be able to 
investigate the process in a modern QCD-based framework. We formulated this in a model with 
an effective pomeron flux in the proton, f]p/p{xip,t), and parton distributions in the pomeron 
fq^g/]p{z,Q'^), such that cross-sections for hard diffractive processes could be calculated from 
the convolution, da ~ fp/p fq,g/]p fq,g/p QCD) of these functions with a perturbative 

QCD cross-section for a hard parton level process. This enabled predictions of diffractive jet 
production at the CERN pp collider and also of diffractive deep inelastic scattering. Although 
this seems quite natural in today's QCD language, it was rather controversial at the time. 

It was therefore an important break-through when the UA8 experiment at the CERN pp 
collider actually discovered ■ hard diffraction by triggering on a leading proton in their Roman 
pot detectors and finding jets in the UA2 central calorimeter in basic agreement with our model. 



The observed jets had normal jet properties and by investigating their longitudinal momentum 
distribution one could infer that the partons in the pomeron have rather a hard distribution 
fq^g/]p{z) ~ z{l — z) and also indications"^ of a superhard component ~ 5(1 — z). 

In spite of this discovery, hard diffraction was not fully recognised in the whole particle 
physics community. It was therefore a surprise to many when diffractive deep inelastic scattering 
was discovered by ZEUS^^ and Hll^ at HERA in 1993 through spectacular events having the 
whole forward detector empty, i.e. a large rapidity gap as opposed to the abundant forward 
hadronic activity in normal DIS events. A surprisingly large fraction ~ 10% of all DIS events 
were diffractive. Moreover, they showed the same dependence as normal DIS, demonstrating 
that they were not a higher twist process but leading twist. 

The diffractive DIS cross-section can be writteiP ^xdQUxpdt = ^ (1 + (1 " vf) ^2^^^ 
where fractional energy loss xjp and four-momentum transfer t from the proton define the diffrac- 
tive conditions. For most of the data, the leading proton is not observed and hence t is effectively 
integrated out giving high-precision measurements of the structure function F^^^\x]p, (3, Q"^) in 
terms of the model-independent invariant variables /3 and xjp. " 

In pp, UA8 has provided more information on diffractive jet production through analyses of 
such cross-section;^'. Several different diffractive hard scattering processes have been observed 
in pp at the Tevatron. Events with jets, W , Z, bb or J /tp have a rapidity gap in about 1% of 
the cases, i.e. an order of magnitude smaller relative rate than diffractive DIS at HERA. 



2 Pomeron approach 

The diffractive DIS structure function can be written F^^^^ (x, Q'^,X]p,t) = f]p/p{x]p, t)-F2^(/3, Q^), 
in terms of a pomeron flux and a pomeron structure function, where xjp Pp/Pp is interpreted 
as the momentum fraction of the pomeron in the proton and [3 ~ Pq,g/piP is the momentum 
fraction of the parton in the pomeron. 

Good fits with data can be obtained, provided that also a Reggeon exchange contribution is 
included. Factoring out the fitted xp dependence, one obtains the diffractive structure function 
F^^'^\f3,Q'^) (or F<f). The dependence is rather weak and thus shows approximate scaling 
indicating scattering on point-like charges. There is, however, a weak log dependence which 
fits well with conventional perturbative QCD evolution. The /3 dependence is quite flat, which 
can be interpreted as hard parton distributions in the pomeron. This is borne out in a full next- 
to-leading order QCD fit giving the parton distributions with a dominant gluon component. 

Using this model with diffractive parton densities from HERA to calculate diffractive hard 
processes at the Tevatron, one obtains cross-sections which are about an order of magnitude 
larger than observed. This problem can be cured by modifications of the model, in particular, 
by introducing some kind of dampinj^at high energies, such as a pomeron flux 'renormalization'. 
It is, however, not clear whether this is the right way to get a proper understanding. 

Thus, there are problems with the pomeron approach. The pomeron flux and the pomeron 
parton densities do not seem to be universal quantities. They cannot be separately well defined 
since only their product is experimentally measurable. Moreover, it may be improper to think of 
the pomeron as 'emitted' from the proton, because the soft momentum transfer t at the proton- 
pomeron vertex imply a long space-time scale such that they move together for an extended time 
which means that there should be some cross-talk between such strongly interacting objects. In 
order to investigate these problems, alternative approaches have been investigated where the 
pomeron is not in the initial state, i.e. not part of the proton wave function but an effect of the 
QCD dynamics of the scattering process. 




Figure 1: (a,b) Conventional string configuration and after a soft colour-octet exchange (dashed gluon line) 
between the remnant and the hard scattering system resulting in a rapidity gap. (c) Distribution of the size 
Aj/max of the largest rapidity gap in simulated HERA DIS events at parton level (after matrix elements and 
parton showers) and at hadron level after standard hadronisation (dashed) and when including the SCI model 
with soft gluon exchange probability P = 0.5 or 0.1 (full, dotted), (d) Rescattering model^^— of a large qq or qqg 
system on the target via 'instantaneous' longitudinal gluon exchange. 

3 QCD-based approaches 

A starting point can here be the standard hard perturbative interactions, since they should not 
be affected by the soft interactions. On the other hand we know that there should be plenty of 
soft interactions, below the cut-off Qg for perturbation theory, because ag is then large giving 
a large interaction probability {e.g. unity for hadronisation). A simple, bu t ph enomenologically 
successful attempt in this spirit is the Soft Colour Interaction (SCI) model^^ DIS at small x is 
typically gluon-initiated leading to perturbative parton level processes as illustrated in Fig. ^ 
The colour order of the perturbative diagram has conventionally been used to define the topology 
of the resulting non-perturbative colour string fields between the proton remnant and the hard 
scattering system, such that hadronisation produces hadrons over the full rapidity region. 

We have, however, proper theory only for the hard perturbative part of the event, which is 
separated from the soft dynamics in both the initial and the final parts by the QCD factorization 
theorem. This hard part is above a perturbative QCD cut-off Qq ^ IGeV^ with an inverse giving 
a transverse size which is small compared to the proton diameter. Thus, the hard interactions 
can be viewed as being embedded in the colour field of the proton and hence one can consider 
interactions of the outgoing partons with this 'background' field. A soft gluon exchange can 
then rearrange colour so that the hard scattering system becomes a colour singlet and the 
proton remnant another singlet (Fig. ^ hadronising independently of each other with a rapidity 
gap in between. The gap can be large because the primary gluon has a small momentum fraction 
Xo, leaving a large momentum fraction {1 — xq) to the remnant which can form a leading proton. 

The soft gluon exchange is non-perturbative and hence its probability cannot be calculated. 
The model, therefore, introduces a single parameter P for the probability of exchanging such a 
soft gluon between any pair of partons, where one of them should be in the remnant representing 
the colour background field. Applying this on the partonic state, including remnants, in the Lund 
Monte Carlo generators Lepto for ep and Pythia for hadronic interactions, e.g. pp, leads to 
variations of the string topologies and thereby different final states after hadronisation. 

One should realize that gap-size is a highly infrared sensitive observable (Fig. ^). At the 
parton level, even after perturbative QCD parton showers, it is quite common to have large 
gaps. Hadronising the conventional string topology leads to an exponential suppression with 
the gap-size, i.e. a huge non-perturbative hadronisation effect. Introducing the soft colour 
interactions causes a drastic effect on the hadron level result, with a gap-size distribution that 
is not exponentially suppressed but has the plateau characteristic for diffraction. 

Selecting the gap events in the Monte Carlo one ca n ex tract the diffractive structure function 
and the model (choosing P « 0.5) describes quite well^^the main features of F^^^\x]p, (3,Q'^) 
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Figure 2: (a) Tevatron data on the ratio of difTractive hard processes to all such hard events, compared to the 
SCI model, (b) Ratio of diffractive to non-diffractive dijet events versus momentum fraction x of the intera cting 
parton in p, with CDF data compared to the Pompyt pomeron model, default Pythia and the SCI modei- 



observed at HERA. Using exactly the same model applied to pp at the Tevatron, one obtains the 
correct overall rates of diffractive hard processes as observed at the Tevatron (Fig. [2j . Differential 
distributions are also reproduced^^ as exemplified in Fig. IShi which also demonstrates that the 
pomeron model is far above the data and Pythia without the SCI mechanism is far below. 

The phenomenological success of the SCI model indicates that it captures the most essential 
QCD dynamics responsible for gap formation. It is therefore interesting that recent theoret- 
ical developments provide a basis for this model. The QCD factorization theorem separates 
the hard and soft dynamics and is the basis for the definition of the parton density functions 
fq/N ~ /rfx-e-^^sp+^-/2(Ar(p) |^(2;-)^+T4/[x-;O]'0(O) I A^(p))^+=o- Here, the nucleon state 
sandwiches an operator including the Wilson line II^[x^; 0] =Pexp ig dw~ A^i^Q^w' ^d_i)ta 

which is a path-ordered exponential of gluon fields. The physical interpretation becomes trans 
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parent if one expands the exponential giving 
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with terms of different orders in the strong coupling g. The first term is the scattered 'bare' quark 
and the followin g te rms corresponds to rescattering on the target colour field via 1,2. . . gluons. 
This rescattering^ has leading twist contributions for longitudinally polarised gluons, which are 
instantaneous in light-front time = t + z and occurs within loffe coherence length ~ 1 /rUpX sj 
of the hard DIS interaction. 

This implies a rescattering of the scattered quark with the spectator system in DIS. Although 
one can choose a gauge such that the scattered quark has no rescatterings, one can not 'gauge 
away' all rescatterings with the spectator systemM^ The sum of the couplings to the gg-system 
in Fig. ^ gives the same result in any gauge and is equivalent to the colour dipole model in the 
target rest system (discussed below). Thus, there will always be such rescatterings and their 
effects are absorbed in the parton density functions obtained by fitting inclusive DIS data. 

This has recently been used^^ as a basis for the SCI model. A gluon from the proton splits 
into a qq pair that the photon couples to (Fig. ^) . Both the gluon and its splitting are mostly 
soft since this has higher probability {g{x) and as). The produced qq pair is therefore typically 
a large colour dipole that even a soft rescattering gluon can resolve and therefore interact with. 
The discussed instantaneous gluon exchange can then modify the colour topology before the 
string-fields are formed such that colour singlet systems separated in rapidity arise producing a 
gap in the final state after hadronisation. Similarly, the initial gluon may also split softly into 
a gluon pair (Fig. ^) followed by perturbative g —* qq giving a small qq pair. Soft rescattering 
gluons can then not resolve the qq, but can interact with the large-size qq-g colour octet dipole 



and turn that into a colour singlet system separated from the target remnant system that is also 
in a colour singlet state. Higher order perturbative emissions do not destroy the gap, since it 
occurs in the rapidity region of the hard system and not in the gap region. 

The rescattering produces on-shell intermediate states having imaginary amplitudedr^ which 
is a characteristic feature of diffraction. This theoretical framework implies the same Q'^,x,W 
dependencies in both diffractive and non-diffractive DIS, in accordance with HERA data. 

Another approach, which has some similarities, is the so-called semi-classical approach!!^ 
The analysis is here made in the proton rest frame where the incoming photon fluctuates into a 
qq pair or a qq-g system that traverses the proton. The soft interactions of these colour dipoles 
with the non-perturbative colour field of the proton is estimated using Wilson lines describing 
the interaction of the energetic partons with the soft colour field of the proton. The colour 
singlet exchange contribution to this process has been derived and shown to give leading twist 
diffraction when the dipole is large. This corresponds to a dipole having one soft parton (as in 
the aligned jet model), which is dominantly the gluon. One is thus testing the large distances in 
the proton colour field. This soft field cannot be calculated from first principles and is therefore 
modelled involving parameters fitted to data. This theoretical approach is quite successful in 
describing the HERA data on F^^^\xip; /3, Q^). 

Yet another approach starts from perturbative QCD and attempts to describe diffractive 
DIS as a two-gluon exchange."*^^ Here, the photon fluctuates into a qq or a qq~g colour dipole, 
which couples to the two gluons as calculated in perturbative QCD giving essentially the (3 
and dependencies of the diffractive structure function xjpF^^^^ = + Fj^g + F^, with 
contributions of qq and qqg colour dipoles from photons with transverse (T) and longitudinal 
(L) polarization. The connection of the exchanged gluons with the proton cannot, however, be 
calculated perturbatively. This soft dynamics is introduced through a parameterisation where 
one fi ts t he xjp dependence, which introduces parameters for the absolute normalization. The 
result is a quite good fit to the data and the different contributions from the two dipoles 
and photon polarizations can be investigated and provide interesting information on the QCD 
dynamics described by this approach. 



4 Gaps between jets and BFKL 

In the processes discussed so far, the hard scale has not involved the gap itself since the leading 
proton has only been su bjec t to a soft momentum transfer across the gap. A new milestone was 
therefore the observational^ at the Tevatron of events with a gap between two high-p^ jets. This 
means that there is a large momentum transfer across the gap and perturbative QCD should 
therefore be applicable to understand the process. This is indeed possible by considering elastic 
parton-parton scattering via hard colour singlet exchange in terms of two gluons as illustrated in 
Fig. ini In the high energy limit s/\t\ ^ 1, where the parton cms energy is much larger than the 
momentum transfer, the amplitude is dominated by terms ~ [q^ ln(s/|t|)]"' where the smallness 
of as is compensated by the large logarithm. These terms must therefore be resummed leading 
to the famous BFKL equation describing the exchange of a whole glu on l adder. 

This somewhat complicated equation has been solved numericall}^^ This gave the matrix 
elements for an effective 2^2 parton scattering process, which was implemented in the Lund 
Monte Carlo Pythia such that parton showers and hadronisation could be added to generate 
complete events. This reproduces the data, both in shape and absolute normalization, which is 
not at all trivial (Fig. ^ . The included non-leading corrections are needed since the asymptotic 
Mueller- Tang result has the wrong Et dependence. A free gap survival probability parameter, 
which in other models is introduced to get the correct overall normalization, is not needed in 
this approach. Amazingly, the correct gap rate results from the complete model, including the 
above discussed soft colour interaction model. 
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Figure 3: (a) Colour singlet exchange by a BFKL gluon ladder giving a rapidity gap between two jets, (b) Fraction 
of events with a rapidity gap between the jets versus jet-i^T, where DO data is compared to the BFKL colour 
singlet exchange mechanisiw — ' with underlying event treatments: simple 3% gap survival probability, Pythia's 
multiple interactions (MI) and hadronisation requiring a 15% gap survival probability, MI plus SCI model and 
hadronisation (no gap survival factor). Also, asymptotic Mueller- Tang (MT) with an 11% gap survival probability. 



5 Conclusions 

After 20 years, hard diffraction has developed into an important research field, with a lot of 
high-quality data from pp and ep. Theory and models have provided working phenomenological 
descriptions, but we do not have solid theory yet. 

In the new QCD-based models, the pomeron is not part of the proton wave function, but 
diffraction is an effect of the scattering process. Models based on interactions with a colour 
background field provide an interesting approach which avoids conceptual problems of pomeron- 
based models, such as the pomeron flux, and provide a basis for a common theoretical framework 
for all final states, diffractive gap events as well as non-diffractive events. 

Finally, the new process of gaps between jets provides strong evidence for the BFKL dy- 
namics as predicted since long by QCD, but so far hard to establish experimentally. 
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